Mendeley as a Source of Readership by Students and Postdocs? Evaluating Article Usage by Academic Status

This paper explores readership counts provided by the social reference manager Mendeley as a source for usage statistics for scientific papers, based on a sample of 1.2 million documents published in journals from the four disciplines Biomedical Research, Clinical Medicine, Health and Psychology. It is shown that the percentage of papers with at least one user on Mendeley (65.9%) as well as the average number of readers per document (9.6) is quite high compared to the uptake and average activity on other social media platforms. The majority of users are PhD and postgraduate students as well as postdocs. Correlations with citations are overall positive, with reading patterns of PhD students and postdocs being in general more similar to citation patterns than that of other professionals and librarians, which reflects expected usage behavior. Important differences concerning these results can be observed between particular research fields, reflecting the particular usage patterns of certain user groups as well as the general uptake of Mendeley in these fields. Most importantly it is shown that differences between usage behavior of user types cannot be accurately determined, as Mendeley only provides only the top 3 user types per document. Introduction Several decades ago, citation analysis has replaced time-consuming reshelving statistics to measure the use of scholarly journals (Gross & Gross, 1927; Garfield, 1972). However, it captures usage by citing authors who represent only a part of the readership of scientific papers. With the shift from print to online, it has become technically feasible and affordable to measure global readership based on full text access log-files (Merk, Scholze & Windisch, 2009). The advantage of downloads over citations as a proxy for article usage is that they mirror activities by all readers (i.e. not only those who cite), including practitioners, teachers, students and the informed public and appear shortly after a document has been published online (Bollen et al., 2009). However, with the exception of a few open access journals, statistics are not available, as publishers consider them too sensitive to make them freely available. As low usage counts could reflect badly on profits, they are only selectively used for promotional purposes such as displaying the most frequently downloaded papers (Haustein, 2012; 2014). More recently, the introduction of social media in scholarly communication has created a new area of informetrics coined “altmetrics” (Priem et al., 2010; Priem & Hemminger, 2010; Priem, 2014). Although the term can be considered as misleading — as these new social-media based metrics do not provide an alternative but are rather complementary to citation-based indicators (Cronin, 2013; Rousseau & Ye, 2013) — and not much is yet known what the various metrics actually measure, social bookmarking systems and reference managers such as Mendeley, Zotero, CiteULike and BibSonomy seem potentially useful for article readership evaluation. These tools help users to organize scientific literature and simultaneously document articlespecific user numbers and have thus been suggested to provide usage statistics of scholarly documents (Taraborelli, 2008; Neylon & Wu, 2009; Priem & Hemminger, 2010; Haustein & Siebenlist, 2011). Mendeley, founded in 2007 and purchased by Elsevier in 2013, is the largest social reference manager providing readership data with over 2.8 million users. Studies found that high shares of scientific papers are available on Mendeley, e.g., 82% of bibliometric papers (Bar-Ilan et al., 2012; Haustein et al., 2013), 97% of articles published in JASIST (Bar-Ilan, 2012), 92% of Nature and Science (Li, Thelwall & Giustini, 2012) and 80% of PLoS (Priem, Piwowar & Hemminger, 2012). While these studies focused on small sets of papers or specific journals, more recent work such as Zahedi, Costas & Wouters (2013), Mohammadi and Thelwall (in press), Mohammadi et al. (in press) and Haustein et al. (in press) explore larger datasets to allow for more general conclusions on the uptake and usage of Mendeley in different fields of science. Moderate positive correlations between Mendeley reader counts and citations — varying between 0.30 and 0.69, depending on the datasets (Bar-Ilan, 2012; Bar-Ilan et al., 2012; Haustein et al., in press; Li & Thelwall, 2012; Li, Thelwall, & Giustini, 2012; Mohammadi and Thelwall (in press); Mohammadi et al. (in press); Priem, Piwowar, & Hemminger, 2012; Schlögl et al., 2013) — suggest that the two metrics measure similar but not identical impact. This indicates that reader counts and citations reflect different foci when citing articles and saving bookmarks and that both metrics should be considered to form a holistic understanding on how scientific papers influence users (Wouters & Costas, 2012). As Mendeley data provides demographic information of users such as their academic status (Gunn, 2013), it is possible to analyze readership in detail. First studies have shown that a large share of users consists of students and postdoctoral researchers, so that a bias towards young scientists exists (Mohammadi et al., in press; Zahedi, Costas & Wouters, 2013). However, the data provided by the Mendeley API has one important limitation, as it only includes the readership counts for the three most frequently occurring user statuses per document. This paper explores user types in specialties of Biomedical Research, Clinical Medicine, Health and Psychology, and shows that the data restrictions bias results on user types. Methods General Uptake of Mendeley. Data was collected using the captures of the Wayback Machine provided by the Internet Archive (http://web.archive.org) selecting the numbers from the Mendeley website content captured on or around the 15th of each month. Readership Analysis. Readership data was collected via the Mendeley API for 1.2 million papers published in journals covered by both PubMed and Web of Science classified as Biomedical Research, Clinical Medicine, Health and Psychology according to the NSF classification system. Retrieval of Mendeley entries is based on bibliographic information including document titles, author and journal names accounting for errors using the Levenshtein distance and is described in detail in Haustein et al (in press). Citations are based on the Web of Science using full citation counts and an open citation window to ensure comparability to readership counts. Following Zahedi, Costas & Wouters (2013) and Mohammadi et al. (in press), Mendeley user statuses were aggregated into sector types (scientific, educational and professional) and user types (Student (Bachelor), Student (Postgraduate), PhD Student, Postdoc, Researcher (Academic), Researcher (Non-Academic), Assistant Professor, Associate Professor, Professor, Librarian and Other Professional) as shown in Table 1. The percentage of available readership counts was computed on the document level by calculating the share of the sum of status reader counts over the total reader counts. It should be noted that readership statuses are selfreported and it is not known whether they are correct or correctly updated when users change status (e.g. from PhD student to Postdoc). 1 User numbers as reported on the Mendeley website on February 14, 2014 accessed via the Internet Archive, http://web.archive.org/web/20140214110051/http://www.mendeley.com/ Spearman correlations were computed for documents with at least one reader to compare the similarity between cited and read papers independently of the coverage, i.e. the percentage of papers on Mendeley following the method values thus slightly overestimate actual similarity between citing and reading behavior but are not influenced by the particular coverage and thus uptake of Mendeley in the particular discipline.